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I. Introduction 

With few exception*, (be methods used today for supporting marxy- 
ttMnaity oomnuinicatioi) (multicasting) efficiently in computer net- 
works involve routing axes. The basic approach consist* of* establishing 
a routing bee for a groop of routing nodes (renters). Once a rooting tree 
is established for a group of routers, a packet or message sent to all the 
routers in the tree traverses each router and link in the tree only once. 
Multicast routing trees (multicast trees for short) arc being used exten- 
sively for multicast Touting in computer networks and mtcrncU [1), [7], 
[8], and have also been proposed for wireless multi-top networks [2]. 
W. 112]. 

Because a multicast tree provides a single path between any two 
routers in me tree, the mini mum number of copies per packet are used 
to disseminate packets to all the reed vers of a multicast group. For 
a tree of W routers, only N — 1 links are used to transmit die same 
information to bJJ the rooters in the multicast tree In a network with 
point-to-point links; in the case of wireless networks with broadcast 
rinks using a single channel, each member of a multicast tree necdi to 
transmit a packet only once. Using rooting trees is of course far more 
efficient than the brute-force approach of sending the same information 
from the source mdi virtually lo each of the other N - 1 times. An 
additional benefit of using trees for multicast rooting is that the routing 
decisions at each router fn the multicast tree becomes very simple: a 
router in a multicast tree that receives a multicast packet for the group 
over an in- tree interface forwards the packet over the rest of its in-tree 
Interfaces. 

However, multicast treea achieve the efficiency and simplicity de- 
scribed above by forcing a single path between any pair of routers. Ac- 
cordingly,, if multiple sources must transmit inf onrtation to the same set 
of destinations, using routing trees requires thai cither a shared mul- 
ticast tree be used for ail sources, or that a separate multicast tree 
be established for each source, Vising a shared multicast tree has the 
disadvantage that packets are distributed to the multicast group along 
paths that can be much longer than the shortest paths from sources 
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to receiver*. Using a separate multicast tree for each source of each 
multicast group forces die routers that participate in multiple multicast 
groups to maintain an entry for each source In each multicast group, 
which does not scale a3 trie number of groups and sources per group in- 
creases. In addition, because trees provide nurnmal connectivity among 
the members of a multicast group, the failure of any hnk In the tree par- 
titions the group and requires the routers involved to reconfigure the 
tree. 

An ad-hoc network is a packet-switching network bused on wireless 
links for router mterooortectian. The topology of an ad-hoc network 
can be very dynamic due to the mobility of routers and the characteris- 
tic* of the radio channels Although tree-based multicast routing la very 
attractive fur wired networks and the btemet because of hs simplicity, 
we argue that it is not as applicable to ad-hoc networks with dynamic 
topologies. Maintaining a routing tree for the purposes of multicast- 
ing packets when the underlying topology changes frequently can incnr 
substantial control traffic. Furtheriisore, during periods of routing-table 
instability, routers may be forced to stop forwarding packets while they 
wait tor the multicast routing tree to bo reconstructed. This paper fo 
cuscs co multicast communication In ad-hoe networks and presents the 
first genejatization of routing trees Into graphs that have more connec- 
tivity than trees and yet prevent long-term or permanent routing loops 
from occurring. We call these routing graphs multicast meshes. The key 
contributions of this paper consist of proving thai if is possible to estab- 
lish and maintain routing structures for multi-point oommoni cation in 
an sd-boc network, that are far more resilient than trees and can make 
efficiem use of communication fesoutee*, and presenting the first mul- 
ticast routing protocol that uses rooting structures different than trees, . 
without the need to Am flood an entire network or Internet with either 
data packets (like DVMRP or PIM-DM do), or control packets dike the 
Forwarding Group Multicast Protocol (FGMP) does [3]>, 

Section )( introduces the main design principles of the Core-Assisted 
Mesh Protocol (CAMP), which builds and maintain* shared multicast 
meshes, and routes packets from any group source over the shortest 
from source to receivers defined in the group's mesh. Section UJ to 
VIII describe in more detail the creation and rnaimcnance of multicast 
meshes and the techniques used for packet forwarding in CAMP. Sec- 
tion IX described the results of simulation experiments used to study 
CAMP's performance compared to tee-bated multicastiag in a dy- 
namic topology. Section X provides our concluding remarks. 

II. Overview of CAMP 

The Core- Assisted Mesh Protocol (CAMP) is designed lo support 
multicast routing in very dynamic ad-hoc networks with broadcast 
links. It adopts the same basic architecture used in It* multicast [6]. 
A mapping service is assumed to exist thai provides ranters with the 
addresses of groups identified by their names. In the Internet, this ser- 
vice would be provided by the Domain Name System (DNS), for exam- 
ple. Hosts wishing to join a multicast group must first query the map- 
ping service to obtain a group address and then Interact with their local 
routers (which we call rower* here) through [GMP [6] or an equivalent 
host-to-toutcr protocol to request membership in a multicast group. In 
addition to the naming service used by hosts to obtain multicast group 



0-78O3-5417-e«9/$t0.00 ©1999 IEEE. 



754 



PAGE 14/22 ■ RCVD AT 11/3/2004 8:11:37 PM [Eastern Standard Time] ' SVR:USP TO-EFXRF-1/1 * DNlS:8729306 < CSID:9164981074 1 DURATION (mm-ss):08-00 



11/03/2004 17:17 FAX 9164981074 



O'BANION & RITCHEY LLP 



©015/022 



addresses, CAMP assumes the avaflibility of routing mfbnnanco firan 
a urucart routing protocol The only requirement imposed on the roo> 
ing protocol used Is that ti must provide correct distances to known 
destinations within a finite time. 

CAMP differs from most prior multicast rowing protocols in that 
it builds and mnrnalns a multicast mesh for information distribution 
within each multicast group. A multicast mesh is a subset of the net- 
work topology that provides at least one path from each source to each 
receiver In rhe multicast group. CAMP ensures thai the shortest paths 
from receivers to auuiucs (called reverse shortest paths) axe part of a 
group's mesh. Packets are forwarded through the mesh along the paths 
that first reach the routers from the sources, i,c, the shortest paths from 
sources to receivers that can be defined within the mesh. CAMP does 
oot predefine such paths along the mesh. A router keeps a cache of the 
identifiers of (hose packets It has forwarded recently, and forwards a 
multicast packet received from a neighbor If the packet identifier is not 
In its cache and the router has been roJd by at least one neighbor that the 
router is the successor in a reverse shortest path to any group source. 

Because a memrxrioic^ redundant paths to 

any other router in (he Same mean, topology changes arc less likely DO 
disrupt the flow of multicast data and to n^re trm reams trucrion of the 
routing strutture* that support packet forwarding. Figure 1 illustrates 
the differences between a multicast mesh and the cojTcapondJ ng shared 
multicast tree; routers that axe members of the multicast group are dark. 
The multicast mesh and tree shown in the figure include routers thai 
have host receivers, posts that are tenders and receiver*, and routers 
that act only as relays. Router 9 is the last receiver 10 join the multicast 
group, and does so In the multicast mesh through either router / or A; 
Kmsequerrtly, rooter c does not become a member of the mesh. 
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Figure 1 illustrates how data packets are forwarded from router h to 
(he rest of the group in CAMP and in a shared-tree multicast protocol. 
In the figure, solid arrows Indicate the Sow of traffic along the reverse 
shortest path In CAMP, and along a shared-tree in the multicast proto- 
col: dashed arrows indicate overhead traffic due to the broadcast char- 
acrerisrtcs of the conutMnicallon channel used to connoa them. Note 
that CAMP delivers data along shorter paths than a shared-tree mulu- . 
cast protocol. The length of paths incurred in multicasting over ad-hoe 
networks is very important, because longer paths require mom rooters 
forwarding packets. It is also interesting to note that, io this example 
the number of routers receiving the packets sent by h a| least OttCC is 
the same using the multicast mesh or the shared-tree, this shows that us- 
ing multicast trees instead of meshes docs not necessarily reduce traffic 
overhead in ad-hoc networks with broadcast links. 

CAMP extends the basic recciver-Ini dated approach introduced in 
the core-based tree (CBT) protocol (1] for the creation of multicast 
trees to enable (be creation of multicast meshes. Cores are used to 
limit the control traffic needed for receivers to join multicast groups. In 
contrast to CBT, one or multiple cons can be denned for each mesh, 
cares need not be pan of the mash of their group, and routers can Join a 
group even if all associated cores become unreachable. A router sends 
a join request towards ft core if none of Its neighbors are members of 
the group; otherwise It simply announces its membership using either 



reliable or persistent Updates. If cores are not reachable from a router 
(hat needs to join q group, the nntter broadcasts Its Join request us- 
ing an expanded ring search (ERS) that eventually reaches some group 
member. When one or multiple responses are sent back to rhe router, it 
chooses any path to (ho mesh. Routers can leave the group simply by 
advertising the change in group rnambenhip to their neighbors. 

The Forwarding Oroup Multicast Protocol (FCMP) O) also builds 
a variation of meshes. However, to establish group meshes. FGMP 
requires for control packets to be flooded in an ad-hoc network. This 
approach is acceptable only in small networks. In contrast, the use of 
cores in CAMP eliminates the need for flooding, unless all cores are 
unreachable from a connected component. 

CAMP uses a scheme based on the rraiisirussion of heartbeat mes- 
sages to ensure (hat the mesh contains all the reverse shortest paths. 
Each mesh member temr^rarfly keeps track of traffic sources whose 
packets come through members other than their res pec live reverae 
shortest paths to the sources. Whenever such situation arises, a hearv 
beat Is sent CO the Successor in the reverse shortest path to the fonrce 
given by the unicast routing tabic. That heartbeat message triggers a 
push join message when the successor is not a mesh member. The push 
join forces (hat specific successor and all routers in the path to the traf- 
fic source to Join the mesh. Mesh components merge together by means 
of similar push joins sent towards cores. 

The mappings of multicast addresses to (one or more) core addrasca 
are dissenunated from each core out to the network as part of group 
mcnmershlp reports. 

III. RoirrrrtO Information maintained in CAMP 

Each router maintains a routing table (KT) built with the urn* east rout- 
ing protocol OSed in Conjunction with CAMP, a f^r^to-fJrqup Address 
Mapping (CAM) table, an anchor table CAT), a multicast routing table 
(MOT), and a packet-forwarding cache. 

At router », the RT rnade available to CAMP specifics, for each des- 
tination j 1 the successor (4} ) and the distance CO (he destination (iPj). 

Router t's CAM consists of a vector of core-to-group address map- 
pings. Each entry of the CAM specifics a group address and the ad- 
dresses of the cores that can be contacted for that group. 

Router i"s AT has an entry for each of the multicast groups In which 
router t is a mcrober. For each multicast group 0, an entry In the AT 
specifies those neighbors thai router i uses as its anchors for the group, 
and whether the router has any local host that is 4 source or receiver 
of the group. An anchor for router i in groop g is a neighbor router 
that is a successor (next hop) in the reverse sftc«tert path loir/eotf one 
joint* in I he group 9. Each anchor entry consists of the address of the 
neighbor and an age field lu the auurtpte shown in Figure 1, router / 
uses router g as an anchor for the group. Note (hat a router does not 
maintain anchor mformation for each source in a group. 

The MKT at router i lists, for each multicast group address g known 
to router i and for each neighbor n that has advertised being a mem- 
ber of the group, an anchor jlag stating whether « is One ofn's anchors 
in g and a manbenhip type jlag staring whether n Is a simplex or du- 
plex member of the group. A router joins a group in simplex mortn 
If It intends to only send traffic received from specific attached hosts 
or neighbor routers to the rest of the group, and it does not intend us 
forward fjackets from the group. Duplex nuoibership Implies that the 
router forwards any multicast packet for the group. 

The packM-fcTwarriing cache maintains the identifier of packets re- 
cently ftnwajded by (he router. Basically the inforrnation kept in this 
data structure comes straight from the tp packet header; which are 
source address, destination address (group address), packet identifica- 
tion and fragment offset. The address of die neighbor that relayed thai 
packet is also stored The main role of the packet forwarding cache is 
to avoid packet replication by keeping track of packets already received 
by the muter. 
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When router ?9 MRT CT AT is updarcd, it sends a multicast routing 
update (MRU) to all its neighbors reporting change* In les group mem- 
bership and anchors per group. An MRU contains one or more entries, 
end each entry spodtles: 

• A group address. 

• An operation code specifying a quit nctlfl cation, simplex mem- 
bership notification, or a duplex membership notification. 

• lo membership notifications, the list of anchors Deeded by router 
i for the group, 

la an ad-hoc network, Changes In nullic*K-group memberships 
should be tissembiaied together with routing-table updates, and rooters 
hear the reports from (heir neighbors and remember which neighbors 
belong to which group, To save bandwidth, routers should exchange 
multicast routing Iriforrnatinn together with their unicest routing-table 
updates. Hence, a routing-took update would consist of a unicast part 
and a multicast part. However, we describe CAMP indcrjendcnUy of 
the bnicasi routing protocol with which li Is used. 

The main objective of communicating anchor information among 
routers is to prevent router* etiat are required by their neighbors to for- 
ward multicast packets, from leaving groups prematurely. 
. A router may update its MRT or AT after topology, changes and 
. messages received from Us neighbors. The messages that may nigger 
an MRU are MRUs from neighbors that change group roaribcrshipa, 
ACKs that change both membership and anchor information- 
. Detecting the failure or addition of a link to a neighbor is pan of 
the routing protocol used LA Conjunction with CAMP. For CAMP to 
work correctly. It Is necessary for the associated routing protocol to 
work correctly in the presence of router failures and network partitions. 
Thi« implies that CAMP cannot be used in conjunction with a routing 
protocol based on die distributed Bcllmnn-Ford algorithm jnch as the 
routing protocol of Ihc DARPA packet radio network t J 1]. However, 
(here are several reccar examples of routing protocol b that can be used 
in conjunction with CAMP [5}, [13], TJ4), [16]. 

IV. Basic Joining and QurrriNO Mechanisms 

We assume in this section thai each router can reach at least one 
core of (he multicast group that the router raced* to join, and (hat the 
multicast mesh Is connected. 

CAMP uses a rtccivcr-irri riaTnri method for routers to join multicast 
groups that olden from CBTTs join mechanism in a number of ways. 
A host first determines the address of the group it needs to join aa a 
receiver. The host then uses that address to ask its attached router to 
Join the multicast group. Upon receiving a boat request to join a group* 
the router then determines whether to announce its membership In the 
group or to rtqucsi being added to the group,, and usgs us CAM 10 select 
the core towards which the join request may be sent* fn CBT, joining 
a group always implies a request to join, and a route/ selects the relay 
of a join request as the neighbor router along me shortest path to the 
group core. 

If a router joining a group has multiple neighbors thai are duplex 
members of the multicast group, then the fouler simply changes Its 
MRT and tirectly -announces to iu neighbors that it is a new rncnv 
ber of the multicast group using an MRU. The announcements slates 
whether the router la a simplex or duplex member If MRUs are sent 
reliably (depending ontheunicosc routing protocol) the neighbor nodes 
acknowledge the join aruicomc^ntent. If MRUs are sent unreliably, the 
join announcement is sent periodically, so that neighbors learn about 
unjoin ova time. 

ir a roooT joining a group has no neighbors that are members of the 
multicast groupi then it selects its successor to the nearest core as the 
relay for the join request After mo router selects a relay, it sends a join 
rzqu&ti to all its neighbors. A join request specifies the intended relay, 
the address of the multicast group that tho sending router needs lojoin, 
and whether the router wants to Join In simplex or duple* mode. 



After sending a join request, a router then waits for the first ao- 
Icnrjwloo^mcnt to its request, and retransmits the request after a requcst- 
timeouL The router persists sending Join requests for a number of times 
(e.g., four) as long as the umcast routing table indicates that there are 
physical paths towards any of the group cores and none of iu neighbors 
are group members. Each rerrortsirusston of a request is addressed to an 
intended relay chosen as described above This is similar to the basic 
niechamsrn used in CBTi however, because data packets flow along dif- 
ferent paths over the multicast mesh appending on the source, there is 
no need to ensure a single loopless path to the chosen cote. This means 
that the use of selected relays towards any core is simply used to limii 
the search from the routers towards the multicast mesh, but being able 
to reach a core is not necessary to join a group. 

Any router thai is a regular member of a multicast group and receives 
a join request for the group is free to transmit a/om acknowUdgmtni 
(ACK) to the sending router. An ACK specifies the sender of the join 
request and the rnulticast group being joined, lb reduce channel traffic, 
the router sped fled as the relay of a join request can be allowed to reply 
first by means of a timeout mechanism after a join request is received. 

When the origin or a relay of a join request receives the first ACK 
to its request or the first ACK to a join request for the same multicast 
group, the router becomes part of the multicast group, In the case of a 
relay, the router sends an ACK to the previous relay or origin of the join 
request, even if that neighbor has already sent an update stating that It 
is a member of the multicast group. 

Receivers use a slightly Afferent procedure to leave a multicast 
group in CAMP than In CBT. A router leaves a multicast group when it 
has no hosts that are members of die group and it has no neighbors for 
whom It is an anchor. 

A router leaving a multicast group Issues * quit notification to iu 
neighbors, who in turn can update dteif MKTs, No adcnowtcdgmctits 
are r eq ues t ed for quit notifications, because jn ccrrlrast co multicast 
routing trees, multicast meshes do not dictate the paths taken by mul- 
ticast packets. Quit notifications arc sent as part of multicast routing 
updates. 

In an ad-hoc network, it is likely that the routers serving as access 
points to the rest of the Internet would servo as cores, because they 
are static and they must be known by the rest of the ad-hoc network 
for Other purposes. An interesting feature of CAMP it that cores are. 
allowed to leave multicast meshes, as long as there arc no routers using 
them as anchors. i.c, as as long as they are needed to provide efficient 
paths for the cfoscmi nation of packets hi the multicast meshes of the 
groups. 

In the example shown in Figure 1, the core router could leave if b 
and d did not use it as an anchor. This would be the case If c joined 
the multicast mesh. An approach Co favor rxxKcote routers as anchors 
Consists Of using a routing protocol that rjrovides multiple paths to the 
same destination, and making CAMP use non-core successors when- 
ever possible. 

V. SIMPLEX JOINS 

If non-member routers were allowed to send packets to a multi- 
cast mesh, the only way to reach the mesh without flooding would be 
through one of its cores. Accordingly, cores could become hot spots 
if multiple non-member sources existed, and the paths followed by the 
packets sent by those sources could bo very inefheient due to router 
mobility in an ad-hoc network. Unlike other protocols that allow non- 
member routers to send packets to a mufti cast tree for dissemination 
within the tree, CAMP requires that the router attached to any source 
of packets for the group Join the multicast mesh, lb avoid the dissem- 
ination of multicasi packets to routers that join a group only to allow 
a source-only host to send packets to the group, CAMP allows routers 
to belong in e multicast mesh in simplest mode* rather man as regular 
members. This characteristic of mesh members is used during packet 
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forwarding to avoid the dissemination of data to sender-only muter*. 

In order to adapt alto to bursty traffic, Ihe router connected to the 
source host does not discard data packets until It raaw aa acknow). 
edgmenl fbf his join request in simp] ox mode. The router encapsulates 
data packets into multiple- copies of its simplex-mode join requests. 
Those encapsulated packets are sent towards the coro aa any other join 
requests. 7b minimize the chanoes of making the core a hot-spot, the 
first raider in the path from the source of data traffic to the cote thai is 
already a member starts forwarding the data pockets itself. In the worn 
case, all routers in the path 10 the care are not mesh members, and the 
coro has to be involved in the packet forwarding while the router closest 
lb the core gets lis acknowledgment to the join request 

Figure 2 Illustrates the benefit* of having members that forward data 
in one direction only. In CAMP, routers m, i, and e Join the group in 
simplex mode, and forward traffic from host A to the rest of the mesh. 
Tn contrast, in a shared-tree protocol, routers m, i, e, and a would for- 
ward packets from host A to router * , the core, and not join the group. 
It is clear from this example that the approach used in CAMP leads to 
shorter delays in the distribution of packets from non-member hosts and 
less congestion at the core routers; furthermore^ because of the a ironies 
memberships in the mesh, traffic from other sources does ooi flow to 
non-member so urc es. Dotted arrows are used again to Indicate over- 
head traffic due to the broadcast characteristics of the coTnmunicatlon 
channel, and solid arrows indicate the traversal of packets accepted at 
each relay and end point. Simple* routers are shown in bold circles. It 
is important to note Mat cores In CAMP do not need to be pan of of a 
multicast mesh. In Figure 2, the mesh could exist even if z were nor pan 
of It because traffic flows along reverse shortest paths and source-only 
pedes* are part of ihc mesh through unidirectional pacha. In contrast, 
in CBT Tor example, the core is contacted by traffic from source-only 
nodes and the sharcd-oxc breaks when the core tails. That is, consider* 
ing the shared -tree example of Figure 2, node z must be a member of 
the shared-crcc if traffic from non-trtemnen is to be propagated to the 
multicast group. 




Rf. r Tttflte Bjw fnm no(K«zx*cr icwcei In CAMP (left) od * sWrf-trcc traded 

(rijbt). 

VI. HEARTBEATS, PUSH JOINS, AND ANCHORS 

CAMP ensures that all the reverse shortest path* between sources 
and receivers are pan of a group's mesh by means of heartbeat and 
push join (PJ) messages. 

Periodically, every single entry In the packet forwarding cache is 
verified. The router looks Up ili RT to chock whether (he ndghbor 
ihat relayed the packet Is the reverse path to the source for every cache 
entry. A heartbeat or a PJ is sent towards every source stored in the 
cache that had the number of packets corning from the reverse path 
under a threshold. 

A router receiving a heartbeat for a given multicast group and «ource 
retransmits the heartbeat Kitt successor towards the origin of the heart- 
beat (determined with the urdcast routing protocol) is aJ ready a mesh 
member. When a member router receives a heartbeat and detects thai 



its successor is not part of the multicast mesh, it sends a push join (PJ) 
to that neighbor router and waits for an ACX from that router. 

Alternatively, if (he reverse-path successor for a source of an ac- 
cepted multicast packet is not a mesh member, the router sends a PJ to 
that neighbor router and waits for an ACK from that router. 

A router retraxismits a PJ after a request timeout, and persists sending 
the PJ, until the unicast routing table Indicates that there Is no path to 
the origin of the heartbeat. A PJ speed to: 

- The multicast group being Joined. 

• The intended relay of the PJ- 

m The end point of the PJ (Le.. the address of a router with an at- 
tached source). 

If an ACK to a PJ is needed from a neighbor and Ihe link to that 
neighbor fails, the router sends a new PJ to a different neighbor using 
the updated information In its unicast routing cable. 

A router that receives a PJ sends an ACK if: (a) it is the intended 
relay, (b) it Is already a member of the group specified in the PJ, and 
(c)it has a path to the end point Of the PJ. A router sending an ACX to 
a neighbor's PJ understands that it Is a group anchor for thai neighbor. 

A router receiving a PJ forwards it (0 the next relay if: (a) it is the 
specified intended relay and (b) it has a path to the end point of the 
PJ. The relay specified in the forwarded PJ is the router's successor to 
the end point of the PJ. A router discards a PJ for which it is not the 
intended relay or for which it Is the Intended relay but has no path to 
the end point of the PJ. 

Heartbeats ire sent as long as the reverse shortest path is quiet and 
anchor information is aged accordingly to account for Ihe fact that 
the reverse shortest paths used for data distribution change over time, 
because sources may leave groups and routers may move frequently. 
When a router stops seeing traffic, it obviously slops forwarding data 
packets. This causes some of the anchors stored at member routers to 
age out. This in turn reduces the number of copies of the same multi- 
cast packets from other sources received by some routers, and may also 
allow soma routers to leave the group. 

After topology changes, the reverse shortest paths from sources to 
members of the group change, which obsolctes some of the anchors 
stored at routers. However, packet forwarding in CAMP depends only 
indirectly on the reverse-path information obtained from the routers' 
unicast routing tables. Anchor information is used only to prevent 
routers to leave a muldcast mesh when they ore in the paths between 
sources and reccrvers in the mesh, and packets flow along the shortest 
paths within the mesh. Accordingly, It is acceptable for a router to at- 
tempt to add anchors as quickly as possible (i.e., as soon as St detects a 
hcartbeal from a router for which the successor is not in the multicast 
meshX and to wait tor anchor information to age out for deletion. 

Router i can add a neighbor p as an anchor for group g in two ways 
after i receives a heartbeat or a push join associated to some source S 

in g: 

■ When p forwards an acknowledgment to the push join and Is also 
the successor for i in the reverse path to source S- When router i 
forwards the acknowledgment, it also sends a multicast update if 
p became an anchor. 

■ When router t gets data packets from router p, which is also a 
successor for i in reverse path to source S. 

Anchors are aged while they are stored in the AT and MKT, and are 
erased when they reach a 0 age. A router can leave a multicast mesh 
when its MKT shows that no neighbor uses it as an anchor and it has no 
attached hosts that are senders or reed vers of the group. 

VII. Handling Topology Chanobs 
A. Link failures 

Link failures are not very critical in CAMP, When » link Tails break- 
ing the reverse shortest path to a source, the router □ Heeled by the break 
may not have to do anything, because the new reverse shortest path may 
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very well be put of the mesh already, ruifhcxmoie, packets keep flow- 
ing along che mesh through the remaining paths to ell receive*!. In 
contrast if any branch of a multicast tree fails, the tree own mean, 
nect all components of the tree tor packet forwarding to continue to all 
receivers. 

link failures have a smaller effect in CAMP than in tree-based mul- 
ticast protocols, because * router joins a group with the first ACK it re- 
ceives from any neighbor, and because a router persists in joining while 
It has neighbors that are members of the mesh or its unicast routing La- 
bi© provides a path to a core. Foithennoftt core futures do not interrupt 
packet forwarding in the mesh or the ability or new members to join a 
group, because ERS can be used to reach a multicast mesh when cores 
are not re achflhle . and cores need not be part of the mesh. In contrast, irt 
tree-based multicast roaring protocols based on receiver-imriaied join- 
ing (e.g>, CBT and PIM-SM), failure of the core or rendezvous point 
of che group breaks the multicast tree, and prevents new members from 
joining, until a new one is elected and made known to all routers. 

A Soda Failures 

CAMP reduces control traffic associated with the establishment and 
maj ntcnance of rautdcast meshes by using multiple cures per group that 
routers can use as landmarks to orient their Join requests. Therefore, 
a router can try to Join a mesh orienting its unicast join requests to 
any of such landmarks and can redirect its join requests when topology 
changes. If none of the cores of a group ore reachable given trie urricasi 
routing information currently available when a router needs to send a 
join request, this router uses an Expanded Ring Search (ERS) to reach 
the mesh. This router first sends a mesh search message specifying 
itself as the requester. Any route/ receiving such a message forwards it 
appending its ID to the path of the message, if the ERS can proceed and 
the router is not a member of the mesh. A muter that receives the mesh 
search message and is a mash member replies with an atiuiowledgmenL 
When the mesh search requester gets the first acknowledgment us its 
message, it sends a Join request along the path U obtained with the 
acknowiedgrrtent. The router retransmits its search message after a time 
out if 1 1 doc* not receive an ACK. 

Hence, CAMP has no single point of failure and can use as many 
cores as desired for a given mesh, in contrast, CBT and PIM-SM re- 
quire a single core to be used In enter to provide A multtcajt tree at all 
(i.e.. avoid to detect loops in die multicast tree and detect partitions). 
ERS could be used when the sfnglo core foils, of course, but that still 
leaves CAMP as a more cOidem" approach, because ER5a are used less 
often by providing multiple cores (no single point of failure). A pro- 
posal to accommodate multiple cores and still provide multicast trees 
has been made recently [IS], but the mechanisms In such a proposal ap~ 
pear much loo complex for a dynamic network and no similar solutions 
have been proposed for ad-hoc networks [4], [121. 

C Keeping Mcjhct Connected 

A multicast mesh may be partitioned due to Che mobility of routers or 
the partition of the network itself, m such a case, CAMP has the abfljiy 
to continue the operation of all mesh components, because routers do 
not rely on a single core to join the mesh. In any uxc-bascd protocol 
based on receiver-initiated joining, the tree component Including the 
core or rendezvous point can continue to operate, but the other must 
terminate the rnulticast group (or make use of ERS. for example, for 
every join request) until a path to the core is re-established 

In addition, CAMP 1* able Id merge mesh components as long as 
there is physical connectivity between mesh components. The mecha- 
nism used to accompliih mis Is simple and is based on requiring each 
router to keep a record of all the cores In a multicast group, even when 
the corns are not teachable. 

When a rooter tOOSes Connectivity with all the Cores of a multicast 
group, it sets a flag to reouuirber to contact any such core when the 



umcast routing table indicates that at least one core for the group is 
reachable. Whcri a rwuer ejects rhattt with at least one core 

of the muldcast group is re-established, it determines if its successor in 
the reverse shortest path to the core is m the mesh, and sends a PJ 
towards the core if the successor is not in the mesh; this use of PJs 
reconnects a multicaat mesh. 

lb ensure that two or more mesh ccenponenta with cores eventually 
merge, all cores that are active in the mesh send periodical messages to 
each other, forcing routers along the path that are not mernbers to join 
die mesh. These messages are core explicit joint (CEJ) chat specify the 
muldcast group, the intended relay of the CET, the intended core, and 
a gap flag. The flag is an information used by the receiver of a CEJ 
to determine whether there ore rion-membeTs m the path between two 
cores. When the flag is kept reset all along the path between the two 
cores, no ackjwvtedgrnent to CEJ needs to be Sent back. 

A router receiving the CEJ with the gap fag set to 0 forward^ the CEI 
to the next relay if (a) it is the specified relay and (b) it has a path to the 
specified core. Rmhermore, if the relaying router is not a member of 
the mesh, It sets the gap flag in 1 in its CEJ. 

A router receiving the CEJ with the gap flag sot to 1 sends an ACK 
if. (a) it is the intended relay, (b) it is already a number of the specified 
group, and (c) it has a path to the specified core. The ACK to a CEJ is 
forwarded all the way back to the core that origliuued the CEJ; ACKs 
force relaying routers tojoin the rricsh asm a PJct^ Alter- 
natively, a router receiving the CEI with the gap flag set to 0 forwards 
the CEJ to the next relay if: (a) it is the specified relay, (b) it has a path 
. to the sped tied core, and (c) it is not a member of the group. 

A similar mechanism Is used to ensure that a connected component 
of a group mesh with no coxes in it can merge itself with at least one 
other mr m r cte d cojnponem with one or mom cores in it. When a router 
that ha group members or is an anchor for other routers delects that 
none of its successors in its shortest path to any core of the group is 
part of the mesh, the router simply sends a PJ towards the nearest core. 

Routers use flooding (ERSs) io reconnect the mesh only if all cores 
are unreachable. 

VIII. packet fqrwardino over a multicast Mesh 

The basic packet forwarding scheme pi CAMP consists of trying to 
forward muldcast data packets along the paths within the mesh chat first 
reach the mem ber routers from the sources. The main control Informa- 
tion in a multicast packet is: 

■ The address of the intended multicast group. 

• The adtfrou of the sending router. 

m A sequence number that 19 used far control (unctions, 

« A time to live used to limit the time each packet Is allowed to 
remain In the network, packet. 

A rooter attached to the source host of a packet simply transmits the 
packet to its ndgfrburs. 

A router receiving a multicast packet without errors from a neighbor 
router accepts the packet only if; 

■ The router is a member of the multicast group speed fied in the 
packet, which is detennined from the router's AT. 

• If the router is a duple* router, the packet's sequence number Is 
not in the ptrckei-forwarding cache. 

- if a simplex router, tha packet's sequence number is not in the 
packet-forwarding cache and the neighbor sending tho packet Is 
also a simplex router 

When a router accepts & packet, it adds its sequence number and 
the identilier of the source to its packet forwarding cache. This step 
prevents the same packet from being accepted more than once by the 
router, provided that the entries in the cache persist longer than the 
time it lakes for packets to revisit a router. In an ad-hoc network built 
using c omm er ci al radios operating in an ISM band with a data rale of 
I Mbps, our arperiments indicate mat smaO p«£±et-forwsrding caches 



788 



PAffi 18122 ' RCVD AT 1 1/3/2004 8:11:37 PM [Eastern Standard Time] * SVR : USPT0-EFXRF-1/1 1 DNIS:8729306 * CSID:91 64981 074 ' DURATION (mnKS):OWI0 



11/03/2004 17:20 FAX 9164981074 



O'BANION & RITCHEY LLP 



©019/022 



suffice (ovg., listing fewer than 40 entries), because each router receives 
few multicast packets per second (due to limits imposed by channel 
access and pacing of transmissions over mnldpte baps) and a successful 
packet takes much less than a couple of seconds to traverse the longest 
nerwericpath, 

A router that accepts a multicast packet need not forward the packet 
any further. A router forwards an nrccptcd multicast packet only if the 
router is an anchor in the rmjltfcasi group far at least one neighbor* Note 
that routers with source-only hosts attached do not receive multicast 
traffic from other sources In (ha group, unless they have connectivity 
with duplex members. 

Whether a router forwards a packet or not, the router updates Us 
MRT with the fact thai the sending romer belongs to the multicast group 
addressed by the packet. This can allow the router to Job the group 
through a simple arincrtlrtcement if it is required in the future. 

It is important to note a few aspects of CAMS"? packet forwarding 
discipline, camp tends to forward packets along the fastest routes 
from sources to receivers chat can be obtained wftfcin a multicast mesh 
at the time the packet Is betng forwarded. It Unk ajynmietrxes are not 
very large, the shortest paths within a mesh tend to be the same as 
the true shortest paths, because a mesh Js buflt enaurmg that aU re- 
verse shortest paths are pan of the mesh. A rare case In which packet 
forwarding would not take place along the shortest paths of the mesh 
would occur when a given router is not an andtcr for any neighbor and 
yet is pan of the shortest path within the mesh from some source to 
some rcccrverts). ' 

IX. Performance Comparison 

A. Protocol Used for Comparison 

In large ad-hoc networks, no multicast protocol proposed to dace 
that is based on render-initialed Joining Is scalable with the number 
of nodes in the network or the number of sources and groups in the 
pcfwodc. Examples of this type of protocol* based en routing trees arc 
DVMRP and PIM-DM; an example of Una type of protocols based on 
graphs other than trees is FCMP [21. The reason that these protocols 
arc not scalable Is that sources must flood either data packets or control 
packets to all the network in order to establish flfOutrng structure. If the 
network size is large. Or the number of groups and sources per group is 
large, this approach is not applicable. 

Accordingly, we focus our comparison of CAMP with other proto- 
cols based on receiver-oriented joining, in which the receivers ask to 
Join the routing structure of a multicast group. Here again, flooding 
of joining reqneits to an entire network u not scalable, and we must 
focus on those protocols that avoid flooding as much as possible, lb 
date, CAMP is the only multicast routing protocol not based on trees 
that avoids flooding of data or control packets to establish the routing 
structure for a group. Therefore, for comparative purposes, we imple- 
mented s simple axe-based protocol that can he used to capture all the 
Features of the main tree-based rroiUicaitprtxceols with receiver-based 
Joining proposed or implemented 10 date. 

We Implemented a shared-lite multicast routing protocol Chat is sim- 
ilar to C8T in (hat it uses a single core and uses that tree to forward 
packets. A router in this protocol, which we denote by WTP (wireless 
tree-based protocol), forwards data packets only when chcy come from 
one of the children or parent of the router in the tree rooted at the core. 
The tree maintenance part of WTP extends the conventiooal shared-tree 
protocols like C0T and PIM-SM. In WTO, a router re-escabUshes its 
connection to the irec by looking for a new parent as soon as It detects 
that «* previous parem has moved away. 

lb compare CAMP against multicast protocols based on source trees 
(e.g., PIM-DM), without taking into account the flooding needed to set 
mc tree, we placed the source at the core of the shared tree In some 
experiments. 




Flf.3. Network fctpoteiy used In etptriaemr Uaacaiu^rit rM^auk£Ai0U»iiBt^ 
boo afatBmd vilh CUT, all rotten us ituJ»t7* of the trohiexft (TOTt Rome Itf Ip (he 
CMS, md A ttd B SM khjcu. 

B. Experiment* 

CAMP rebuilds meshes at least as last as CBT and PIM can rebuild 
trees and requires storage ovarfaead similar to any protocol based on 
Shared trees. CAMP Is always loopJes&v forwards packets around foiled 
links of a mesh, and is resilient to any core failure and network parti* 
ttans. In amirest. CBT and PtM incur temporary loops when (he uni- 
cast routing tables are Inconsistent, stop packet forwarding to segments 
of the group alter link failures until the multicast tree is rebuilt, and 
are vulnerable to core or rendezvous point failures, fiirthercttrc. in a 
static topology, CAMP delivers packets along the shortest paths defined 
within a multicast mesh, which is built based on reverse shortest paths, 
and PIM dense mode and DVMRP build source trees based on reverse 
shorcest paths as well. Therefore, the paths obtained In CAMP in static 
topologies are similar to those obtained with source trees and can be 
much shorter than the paths obtained with shared trees (e.g., CBT). 

Accordingly, the interesting aspects for a performance comparison 
between CAMP and tree-based multicast protocols are the average de- 
lays and packet losses incurred due to node mobility. We ran a num- 
ber of experiments to study this aspect of CAMP's p e rforma nce and to 
compare it against the tree-based multicast approach. Figure 3 shows 
the topology of the dynamic network used in the simulations. The net- 
work has 30 routers, numbered from 1 to 30, and two senders, A and 
B. Although not indicated in the Pigure, this network has high cocneo 
tivity; routers have an average of six neighbors each. The Nnks shown 
in the diagram illustrate the Initial shared tree computed dynamically 
in the simulation. All nodes In The two multicast routing protocols are 
receivers. In CAMP, this means all nodes are duplex members, Rooter 
16 was chosen as core for all simulations. 

Experiment? ran for 350 seconds and the same conditions were ap- 
plied to the simulation runs for Camp and WTP; sperifically the same 
number of packets was sent from the given source, the same router mo- 
bility was applied, and the same M aC and routing protocols were used. 
The simulations used a single broadcast channel, so that the transmis- 
sion of a node is received by ail its neighbors. The floor acquisition 
multiple access (FAMA) protocol[91 was used to access ma broadcast 
channel, and the wireless internet routing protocol (WIRP) [10] with 
bop cowxi as distance metric was used to generate the unicast routing- 
cable entries at routers. Radio linkt arc bidirectional. The experiments 
were meant to show how much more packet loss is incurred in a tree- 
based scheme than in CAMP 

Two major types of experiment were run regarding mobility: one 
with 15 routers and the other with only 5 routers moving through the 
network. The speed at which mobUe nodes moved randomly In all slnv 
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ulanon* was €7.5 nulcs/hr (30 meters/aec). with the exception of one 
simulation experiment in which 15 routers were mobile at a walking 
speed of 0.135 raUea/ni(0L6nicccrcfecc). 

In most experiments, data traffic Is originated cither by source A, 
which is direcUy attached 10 the cost (/outer 16), or by source B, which 
is auached to muter 29. In one experiment, five routers were mobile, 
and the source B was positioned away from the core and was attached 
CO the moving muter J 7. 

The results for the case in which (he source Is directly attached tp 
(be core arc a good basis for comparison between CAMP and multi- 
cast ranting protocol* based OB source trees, such as P1M and DVMRP, 
because all routers are group members, which eliminates unnecessary 
overhead for such protocols. 

The percentage of packets lost ai a receiver is simply the amount 
of packets sent by the traffic source that was not seen by the specific 
receiver. Therefore, the smaller the percentage is. the better the pro- 
tocol behaves. OSviously, the average packet delay measured ax each 
receiver excludes lost pockets. 

Figure 4 to Figure 7 show the behavior of CAMP and WTP when 
half of the routers move at random rod their speed is only 0.6 tro> 
Cers/sec. Because routers move slowly, the topology or the network 
does not change very drastically and both protocols experience few 
packet losses and small average delays. However, CAMP always pro- 
vides fewer packet losses than Ibe tree-based approach, in Figure 7, the 
overage delays In WTP appear amallcr in CAMP but actually they are 
not This is because delays an measured based on delivered packets, 
and WTP delivers much fewer packets than CAMP, which biases (he 
delay measure. 

Figure 8 to Figure 1 1 show the behavior of both CAMP and WTP 
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for a more dyrurrrric environment in which five nodes move randomly at 
high speed (67 J milcs/hr) and the source does not move. The source in 
all these cases is static and b either directly EHBched to or distant from 
the core. When the source of traffic is directly connected co the core, we 
have the best scenario for WTP, because the shared tree becomes also 
a source-based tree. As the figures show, much fewer packet losses 
occur In CAMP than In (be tree-baaed approach; CAMP delivers at 
least the aamo number of packets to each receiver. The average delays* 
are comparable between CAMP and WTP for the case In which the tree 
Is a source tree, and tend to be bercer in CAMP when the source is away ' 
from the core, even though the sample of delivered packets for CAMP 
is larger than the sample for WTP This is due to the larger number of 
changes to (he tree built by WTP when nodes move at higher speeds. 

Figures 12 and 13 show the performance of CAMP and WTP when 
the source te one of the nodes moving a< highspeed; The negative im- 
pact of the source's mobility in a iiee-bascd approach is evident; while 
In CAMP routers miss less than 5% of the packets, in WTPixraters miss 
at least 12% and si* rowers miss more than 35% of trie packets. This is 
a direct consequence of the feet that a multicast tree breaks easily with 
node mobility and the source of packets is the one forcing the tree to 
change. 

Figures 14 and 15 show the comparison between CAMP and WTP 
when half of the 30 routers are mobile and the source of data traffic is 
static and directly connected to the core of the tnultkaii group. The 
percentage of packet losses in CAMP is less than 15% at aoy receiver 
and is never higher than the loss rate to the tree-based approach; in con- 
trast, loss rates in WTP are at least 60% in 10 receivers and higher than 
40% in more than half half die routers, with only six routers receiving 
the same number of packets as in CAMP. Figure 15 iUnstrates the delay 
ctpcrienrcd by those packets that reach a receiver; a one-to-one com* 
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pflitSOO cannot be made accurately, bOC8U5e CAMP delivers many more 
packets to the majority of Che receivers. However, for those receiver? 
that experience the) same pocket loss in CAMP Mid WTp (routers 4, 5, 
8, 12, 21, and 24) the average dejjrys In both protocols arc almost the 

Recall that in WTP and in any tree-based protocol for ad-hoc net- 
toOfk&i routers must attempt CO reconnect the tree as soon as possible 
every time a router loses its parent in the shared tree. Every time the 
unicast routing protocol warns WTP about a neighbor being removed 
from the umcast routing table, (he protocol sends a join request to the 
new successor to the core, trying to re- establish its connection to the 
tree. Figure 16 shows a much more pronounced performance differ- 
ence between CAMP and WTP. Packet louses in CAMP arc similar to 
the case in which the source and the core are cotocaced, which is 10 be 
expected because the location of core* Is inctcvasx for packet forward- 
ing in CAMP. In contrast, all receivers experience much more pocket 
losBca with the generic shared-tree protocol than they do with CAMP, 
and 24 receivers have packet-loss rates higher than 40%. The reason 
for that poor behavior of WTP la the strong dependency it has on con* 
"Stem onicast routing tables to provide a loop-free shared tree. The 
unicajt routing protocol used In the experiments may create temporary 
loops shortly after links go down. Because WTp makes decisions re- 
garding tree reconoection shortly after links go down, the shared tree 
becomes vulnerable to loops, which leads lo die larger packet-loss rate. 
This fact shows the difRcuLtiai brought up when packet forwarding is 
dictated by a strict delivery structure like a Shared tree in a dynamically 
changing environment. Protocol behavior in the presence of temporary 
loops in unicast routing also illustrates CAMP's survivability. 

The same problem observed in the generic shared-tree protocol when 
the network ix more dynamic would occur in a mobile extension of 



Rs> 1 I. Avtnpo doty ftf ttolticait rial* pacfadi received ii eocfa route **** Oflly fiwo 
rotfCfi iiicifa st hi^fe bjjuuJ tod lbs lourca of nfftc ti Malic ado Ii tcwy fton die eme 
- of die naulduM graufK 

PIM-SM. i.e., trying to re- establish link branches very frequently (10 
keep packet* flowing) when routing tables ore inconsistent. 

X. Conclusions 

We have introduced the first multicast routing protocol based on a 
routing structure other than trees that docs not require flooding an en* 
tire network to set Up its routing Structure. CAMP consisu Of the main- 
tenance of multicast meshes and toopless pocket forwarding over such 
meshes. Within the multicast mesh of a group, packets from any source . 
in the group are forwarded along the shortest paths defined with the 
mesh from the source to the receivers. CAMP guarantees chat, within 

a finite rime, every receiver of a multicast group has a reverse shortest 
path to each source of the multicast group, which is used to reduce the 
sup-«ptimality of the paths traversed within a mesh compared lo the 
true shortest paths, which could include nodes that are not part of the 
mesh. 

CAMP scales very well, because it docs not require sources or re- 
ceivers to flood the entire network with control ox data packets as long 
as there are cores available. Simulation experiments show that CAMP 
easily outperforms tree-based multicast protocol in dynamic networks. 
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